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Abstract 

The upcoming re authorization of No Child Left Behind (NCLB) will be contentious, especially given the 
size of the act and the role it gives the federal government. When making decisions about reauthorization, it is im- 
portant to consider the early evidence about the act’s implementation and effectiveness. 

The first important finding about NCLB is that states have implemented the act with widely varying stan- 
dards. States are given the right to set proficiency levels for the student achievement tests, and a review of the 
evidence shows that some states have set very high standards and some states have set very low standards. States 
also have the duty of setting levels of proficiency for schools to make adequate yearly progress (AYP); again, there 
is great variation in the decisions states have made regarding these AYP cutoffs, and these differences have affected 
the rate at which schools fall short of AYP. Finally, while NCLB requires all teachers to pass a content test to prove 
they are highly qualified, some states have made it very easy for teachers to pass these tests, while others have made 
it very hard. The variability in the implementation of these provisions is difficult to justify. 

The big question is whether NCLB is causing student achievement to increase. The answer is not clear, but 
the evidence is more positive than negative. There is a mild upward trend in mathematics achievement since the act 
was implemented, and this trend is occurring at a greater rate than it was before 2002. Still, there is concern that 
achievement overall is going up while students at the top and the bottom of the distribution have made no gains. 
This may be due to the AYP cutoffs, which reward schools for moving the students right around the cutoff, not the 
students at either end of the distribution. There is evidence of very minor progress toward closing the achievement 
gaps since NCLB was first passed. NCLB holds schools accountable for their students’ learning, a strategy which 
may help raise achievement. However, if the system were to take all students into account by measuring achieve- 
ment gains, it may help make the system even more effective. Teacher quality does not appear to have changed 
much since NCLB; low-income students are still more likely to have unqualified teachers than high-income stu- 
dents. While school choice is included in the act, the evidence shows that few have taken advantage of the offer of 
school choice, in part because parents are often notified too late to make a decision. 

Overall, the evidence on the act is mostly neutral to positive, and there is little evidence that the negative 
effects of the act some predicted have materialized. For reauthorization, lawmakers may consider the following 
suggestions: taking student achievement gains (value-added to student achievement) into account rather than just 
student achievement levels; evening out some of the variation in proficiency standards, teacher requirements, and 
content standards across states; discussing national standards; revising AYP targets to make the 2014 goal more 
reasonable; holding students accountable for their achievement; and providing supplemental services that are shown 
to be effective through educational research. With these changes, the act should continue to help teachers improve 
student achievement though high quality instruction and a rigorous curriculum. 
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This SPR presents an article by Porter and Polikoff summarizing research on the implementation and 
effectiveness of the No Child Left Behind (NCLB) legislation. The article is based on material in a new book, 
Standards- Based Reform and the Poverty Gap: Lessons for “No Child Left Behind ”. 

It was Marion Wright Edelman of the Children’s Defense Fund who originally coined the phrase, “leave 
no child behind.” It is now somewhat ironic that the most controversial and contentious legislation affecting 
children in recent years bears this name. On the other hand, it is very reassuring that we have this body of re- 
search evidence to guide the pending reauthorization. 

Porter and Polikoff make several important points. First the variation across states in implementation 
and in the standards they set both for student achievement and teacher qualifications is great. Their first recom- 
mendation for reauthorization is that this variation in proficiency standards, and teacher and content standards 
be reduced. Similar variation exits across states in child care standards and Head Start quality. While local con- 
text demands some flexibility, legislation such as NCFB has tended to allow sufficient flexibility across states 
that no common thread remains in implementation. 

Porter and Polikoff provide good news in that the possible negative effects of this legislation, which 
many feared, do not seem to have materialized. Instead they report neutral or small positive gains in student 
achievement. They recommend, however, that reauthorization focus on student achievement gains in addition 
to achievement levels, setting national standards and resetting 2014 goals, holding students as well as teach- 
ers accountable, and providing a variety of supplemental services. While this is good news overall, we have to 
interpret it very carefully. Students may show no losses in the measured areas, but we also know that areas not 
touched by NCFB such as social studies, and art and music may have been abandoned by many school districts 
in their frantic pursuit of NCFB standards. 

NCFB is massive legislation with far reaching implications for children and their education. It is also 
better legislation than many of its opponents would admit. However, its implementation and most attention to it, 
including research, has focused on student achievement in core academic subjects and teacher qualifications, as 
summarized in this SPR. The legislation is far more comprehensive than that. For example, it contains a small 
section on character and civics which has led to funding by the Department of Education for a large number 
of character and civic education programs across the country. As a member of APA’s Committee on Children, 
Youth and Families, I have worked with their legislative affairs office to promote attention to this aspect of the 
legislation. Nonetheless, there has been much less research in this area than on achievement. In fact, some fear 
that many school districts have abandoned social studies and civics because it is not seen as part of the core of 
NCFB, for which their performance determines their funding, but there is little data to determine if this is the 
case. 

Brooke and I really hope that the research reported in this SPR will inform reauthorization, which has in 
fact already begun. Furthermore, we hope it will serve as a model for a more comprehensive implementation of 
the new legislation, pursuing more of a “whole child” approach which more than fifty years of developmental 
research has told us is always the way to promote the best interests of the child. 

Fonnie Sherrod, Ph.D., Editor 
Fordham University 
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Policymakers and education researchers are contemplating 
the reauthorization of the No Child Left Behind Act (NCLB). Given 
the controversy surrounding NCLB’s design and implementation, the 
debate will be vigorous. The act represents the most significant pres- 
ence of the federal government in education. Thus, reauthorization 
debates must carefully weigh the evidence available on implementa- 
tion and initial effectiveness of the act. 

NCLB is based on a theory of change that politicians hope 
will raise American students’ achievement. First, create content stan- 
dards to show teachers what they are supposed to teach. Next, use 
aligned student achievement tests based on school-level achievement 
to convince teachers to teach the content in the standards. To ensure 
that all students are taught worthwhile content, use disaggregated 
data — data for specific subgroups of students — for accountability. 
And to ensure that teachers are up to the task of teaching the content, 
set minimum standards for teacher qualifications. 

By design, NCLB requirements have been phased in over 
time. For example, while new teachers were required to be highly 
qualified in 2002/03, the first school year in which all teachers were 
required to be highly qualified was 2006/2007. In 2006, all students 
in grades 3-8 were required to be tested in mathematics and read- 
ing — science testing at these grade levels is not required until 2008. 
Arguably the most controversial provisions of NCLB involving 
school accountability have yet to be fully implemented. NCLB re- 
quires that schools failing to meet adequate yearly progress (AYP) 
achievement targets for six consecutive years must be restructured. 
Given the phasing in of provisions, NCLB is young to be making 
a summative evaluation of its effects. Still, there is some relevant 
evidence from which tentative conclusions can be reached and sug- 
gestions for revision made. 

The following report is divided into three sections. The first 
section describes how NCLB has been interpreted by states in terms 
of proficiency levels, AYP targets, and highly qualified teachers. Next, 
the effects of the various provisions of the act are estimated. This 
section focuses on student achievement levels, the achievement gap, 
school accountability, highly qualified teachers, supplemental ser- 
vices, and student accountability. The final section offers suggestions 
for the reauthorization of NCLB in light of the preceding analyses. 



1 This text is largely based on a chapter from the book Standards-Based Reform and the Pov- 
erty Gap: Lessons for “No Child Left Behind” (Porter, 2007). 

3 




NCLB Implementation 

NCLB, or Title I of the U.S. Department of Educa- 
tion, had a budget of approximately $13.3 billion of the 
department’s $56 billion in 2006 (slightly less than one 
half of one percent of the federal government’s $2,709 
billion in total outlays). Still, Title I accounted for ap- 
proximately 8% of the total funding for K-12 education. 
Title I is a funding stream: the law sets requirements of 
states in order to receive funding, but states are allowed 
considerable latitude in implementing most parts of the 
law. As a result, there are important state differences in 
the implementation NCLB’s requirements. 

Proficiency Levels 

A primary goal of 
NCLB is ensuring that 100% 
of students are proficient by 
2014. Along the way, schools 
are required to meet AYP tar- 
gets based on the percent of 
students of various subgroups 
reaching proficiency. The goal 
is for all students in all schools 
across the nation to reach high 
standards of achievement on challenging academic stan- 
dards. However, states are given considerable discretion in 
setting proficiency standards. The result is that proficiency 
standards vary dramatically among states, bringing into 
question the notion that NCLB really requires leaving no 
child behind. 

Since we know states differ in their performance on 
the National Assessment of Educational Progress (NAEP), 
we cannot draw conclusions about state standards for 
proficiency based only on states’ reports of percent pro- 
ficient — some of the variation is attributable to student 
achievement (Linn, 2005). Arecent report by the National 
Center for Education Statistics (2007) estimated NAEP 
scores required to reach state proficiency cutoffs and 
found remarkable results. In terms of level, for 4th and 
8th grade reading no state in the nation had a proficiency 
cutoff that corresponded to a NAEP score above the NAEP 
proficiency cutoff. For math, just two states at grade 4 and 
three at grade 8 had proficiency cutoffs that translated to 
NAEP scores at or above the NAEP cutoff. In terms of vari- 
ability, while several states’ (Wyoming, South Carolina, 
and Massachusetts) 4th grade reading proficiency cutoffs 
translated to scores just below NAEP’s proficiency cutoff, 
at least 2 1 states set their proficiency cutoffs so low that 
they translated to NAEP scores below the “basic” level. 
States have set their proficiency levels from roughly com- 
parable to NAEP’s to considerably less challenging than 



NAEP’s. NAEP is not a high stakes test and surely is not 
aligned equally well across states, which may partially 
explain the differences between state and NAEP standards. 
Still, it is clear that the proficiency standard does not have 
the same meaning across states. 

Adequate Yearly Progress 

NCLB also leaves to state discretion the setting of 
AYP targets for school accountability. States are allowed 
to vary a) the trajectories they set in moving to the 100% 
proficient goal, b) the minimum group size required for 
reporting disaggregated subgroups, and c) whether to 
use confidence intervals in deciding whether targets have 
been met. As with proficiency levels, it appears that giving 

states discretion has resulted in 
widely varying implementa- 
tion. 

We might expect that 
states that set lower proficiency 
levels might also set easier- 
to-reach AYP targets; surpris- 
ingly, however, there appears 
to be almost no correlation 
between the rigor of a state’s 
proficiency standards and the 
percent of its schools failing AYP. North Carolina had 
among the least challenging proficiency standards and 
42% of its schools failed AYP, while Texas had almost 
the same percent proficient and 10.7% of its schools failed 
AYP. Porter, Linn, and Trimble (2005) examined the ef- 
fects of state decisions about AYP targets on the percent 
of schools failing AYP using Kentucky data. As a result 
of Kentucky’s decisions, just 6% of schools failed AYP in 
2004. If Kentucky had used different decisions, such as 
shrinking minimum group size for disaggregation or using 
a straighter trajectory toward 100% proficiency, as many 
at 56% of Kentucky schools would have failed AYP that 
year. In short, states vary in how they set AYP targets, but 
over time most states have moved to progressively more 
lenient targets. 

Highly Qualified Teacher 

NCLB emphasizes teachers’ importance in raising 
student achievement through its highly qualified teacher 
requirements. However, here again the act leaves consider- 
able discretion to states. By 2005-2006, all teachers, with 
few exceptions, were required to have a bachelor’s degree, 
have certification, and have proven content knowledge of 
the subject they teach. States differ in their certification 
requirements and in their definition of “proven” content 
knowledge. 

Thirty-seven states use the Educational Testing 



...there are important state 
differences in the implementation 
of NCLB’s requirements. 
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Service’s PRAXIS II test in secondary mathematics to 
determine the content knowledge of their secondary math- 
ematics teachers. The national 25th percentile score on this 
test is 128, the median is 143, and the 75th percentile is 
157. State cut scores vary widely — from 156 in Colorado 
to 116 in Arkansas. The middle school mathematics test 
has similar variability, with a median of 161, a maximum 
of 163 in Virginia, and a minimum of 139 in Nevada and 
South Dakota. It is clear that, while the policy of verifying 
teachers’ content knowledge is a national one, states have 
implemented the policy quite differently. Being “highly 
qualified” does not mean the same thing across states. 
Given the importance of teacher quality in student achieve- 
ment, these seemingly arbitrary state-to-state differences 
in what it means to have proven 
mathematical content knowl- 
edge are unsettling. 

Preliminary 
NCLB Effects 

The evidence above 
shows that, when states are 
given latitude to make deci- 
sions about the implementation 
of NCLB provisions, the result is great variation. This 
finding alone has implications for reauthorization. The 
next section presents early evidence of NCLB’s effects in 
terms of student achievement, the achievement gap, school 
accountability, student accountability, highly qualified 
teachers and supplemental services. 

Student Achievement Levels 

Although it is too early to see NCLB’s full effects 
reflected in student achievement gains, it is useful to ex- 
amine results from the National Assessment of Education 
Progress (NAEP) to see if there have been changes in 
achievement in the years since the act was first autho- 
rized. Overall, NAEP results reveal positive changes in 
mathematics but no real change in reading achievement. 
Since 2000, mathematics scores on NAEP have jumped 
considerably after a decade of relative stagnation. In 4th 
grade math, the percent of students at or above proficient 
has jumped from 24% to 36%, and in 8th grade math the 
percent has risen from 26% to 30%. Long-term trend 
NAEP also reveals a growth in mathematics achievement 
among 9 and 13 year-olds since 1999. In reading, 4th grade 
achievement declined slightly during the 1990s before 
rebounding to earlier levels in 2002 and 2005. Among 8th 
graders, reading scores have remained essentially constant 
since 1992. Long-term trend scores show similar results, 
with 9 year-olds gaining slightly since 1999 and 13 year- 



olds remaining flat. 

A recent report from the Center on Education 
Policy (2007) suggests an even more positive effect of 
NCLB on achievement. Using state reports of percent of 
students scoring proficient and effect sizes, the authors 
found that substantially more states showed improvement 
in student achievement since 2002 than showed declines. 
The report also suggested that gains seemed to be the 
greatest at elementary levels and in mathematics, but that 
gains outstripped losses even in high school mathematics 
and reading. Furthermore, in states that had been under- 
going standards-based reform prior to NCLB, the pace 
of improvement in student achievement seemed to pick 
up with NCLB’s implementation in 2002. Taken together 

with the NAEP findings above, 
there is cause for cautious 
optimism about NCLB’s ef- 
fects on achievement. Though 
these results cannot be exclu- 
sively attributed to NCLB, and 
though the full effects of the 
act may not have been seen, it 
is a hopeful sign that achieve- 
ment has been on the rise in 
recent years. 

At the same time, there is some concern that, 
while NCLB is raising average achievement levels, it is 
doing so at the expense of students at the top and bottom 
of the achievement distribution. The theory is that, since 
adequate yearly progress is defined by percent proficient, 
schools and teachers are only motivated to work hard to 
improve achievement among those students just below 
the proficiency cutoff. A recent study using data from the 
Chicago Public Schools (Neal & Schanzenbach, 2007) 
suppports this hypothesis. This finding causes concern, 
but it may be addressed by considering the value-added 
approach described later. 

NCLB and the Achievement Gap 

One of the primary focuses of NCLB is the elimi- 
nation of achievement gaps. The act seeks to eliminate not 
only the racial/ethnic group achievement gap, but also gaps 
by gender, English proficiency, migrant status, poverty 
status, and disability status. Under the current provisions 
of the law, all of these groups will have to be 100% profi- 
cient in each school (given sufficient numbers) by 2014. 
Understanding the challenge requires understanding the 
size of the gap and how it has changed. 

The racial achievement gap is large and robust no 
matter the age group or academic subject. The gap is pres- 
ent at an early age — the Black- White gap is 1.2 standard 
deviations on the Peabody Picture Vocabulary Test at ages 



Being “highly qualified” does not 
mean the same thing across states. 
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three and four (Jencks & Phillips, 1998). The gap also may 
grow somewhat during the K-12 educational experience, 
by as much as one-third of a standard deviation (Phillips, 
Crouse, & Ralph, 1998). One of the best tools for exam- 
ining the achievement gap over time is long-term trend 
NAEP. The gap on this assessment appeared to narrow 
most during the late 1980s, remaining relatively stable 
since then (Hedges & Nowell, 1998). The Black-White 
gap currently stands at roughly .7 standard deviations in 
reading and .9 standard deviations in math for 13 year-olds. 
The Hispanic- White gap stands at roughly .7 standard 
deviations in reading and .8 standard deviations in math 
for 13 year-olds. No conclusive changes in the gap can be 
seen in the years since NCLB was initially authorized. 

Another important achievement gap that NCLB 
seeks to close is the disability gap. NCLB requires schools 
to increase proficiency for all students regardless of dis- 
ability. This breaks with a long-held tradition in schools of 
special education provisions and requirements being kept 
apart from general education. NCLB and the Individuals 
with Disabilities Education Act work together to ensure 
high quality instruction for all students (Loorman, Kalin- 
owski, & Sexton, 2007). These provisions go a long way 
toward bringing special education into the mainstream 
by requiring high standards of all students and equalizing 
student opportunity to learn. 

A third gap of concern is the SES achievement 
gap. The size of this gap depends on how SES is defined. 
Sometimes, the achievement gap is defined in terms of 
family income. Other times, such as on NAEP, the clas- 
sification used is parents’ education level. The NCLB 
definition classifies students based on their receipt of free 
and reduced price lunch. As a result of these different 
definitions, it is difficult to come to a single estimate of the 
SES achievement gap. Using parents’ education, long-term 
trend NAEP reveals that the gap in reading is about .6 of 
the difference between achievement levels, and that it has 
remained almost unchanged since 1978. In mathematics, 
the gap has narrowed slightly to about .6 of the difference 
between achievement levels, due primarily to an increase 
in achievement among students whose parents did not 
graduate high school. Similar to the racial achievement 
gap, several studies have shown that the SES gap does 
not increase at all during the school year (Entwisle & Al- 
exander, 1992; Heyns, 1978) and that summer losses are 
greater for students from low SES families than students 
from high SES families (Cooper, Nye, Charlton, Lindsay, 
& Greathouse, 1996). These findings reveal that schools 
may not be primarily responsible for the creation of the 
achievement gap. The question NCLB asks is “What can 
schools do to close the gap?” 

There are many strategies for closing the achieve- 



ment gap, but researchers have identified six of the most 
common: pre-school programs, student ability grouping, 
instructional interventions for students at risk of failure, 
racially matching students and teachers, selection of higher 
quality teachers, and smaller class sizes (Ferguson, 1998; 
Porter, 2005). Of these, the most promising strategies are 
those that address inequalities in student opportunity to 
leam. For instance, students of color and students from low 
SES families are less likely to be taught by high quality 
teachers than are white or more affluent students. Research 
has shown that teacher quality has consistently strong ef- 
fects on student achievement gains (value-added to student 
achievement). NCLB attempts to address the inequity in 
teacher quality distribution through its highly qualified 
teacher requirements. Another example of inequality in 
student access to learning is that students from low-SES 
families are less likely to be taught ambitious academic 
content aligned to rigorous state content standards. NCLB 
addresses this issue by providing choice to students in 
Title I schools that fail AYP for two consecutive years 
and supplemental services to low-SES students in Title I 
schools that fail AYP for three consecutive years. These 
provisions are hopeful, and show that NCLB is targeted 
at reforms that may be the most effective for narrowing 
the gap. 

The preliminary evidence on the closing of the 
achievement gaps indicates modest success. The Center on 
Education Policy (2007) reports that far more states have 
seen gaps in percent proficient narrow than widen. This 
is true for the Black- White gap, the Hispanic-White gap, 
and the socioeconomic status gap. The authors emphasize 
that the changes in the gaps since NCLB are generally 
very small — in most cases a few percentage points. As 
more evidence is gathered, the achievement gaps can be 
evaluated more carefully. For now, this evidence suggests 
small but positive effects. 

School Accountability 

No Child Left Behind has been law for only five 
years, so many of the school accountability requirements 
have yet to be enacted, and their effects are yet to be 
seen. For example, 2005/2006 was the first year in which 
achievement testing was required in all grades 3-8. With 
multi-grade testing, even most small elementary schools 
now meet the requirements for disaggregated data. The 
result may be more schools failing AYP. Also, the first 
increases in AYPtargets were mandated in 2005, and these 
increases will continue until 2014. At that point, when 
100% proficiency is required, the number of schools failing 
proficiency could be high. A recent report estimates that 
96% of Illinois schools and 88% of Ohio schools will fail 
AYP in 2014 (Wiley, Mathis, & Garcia, 2005). It is pos- 




sible these estimates may be biased high. Still, there is no 
evidence that schools can actually reach 100% proficient, 
and some scholars have called for more reasonable targets 
(Linn, 2005). 

Researchers have studied state-level school ac- 
countability programs similar to NCLB and consistently 
found evidence of positive effects. Phillips and Flashman 
(2007) found that accountability policies had their larg- 
est effects on high-poverty schools and schools serving 
large proportions of students of color. Carnoy and Loeb 
found that states with stronger school accountability pro- 
grams had greater gains in student achievement for both 
high and low mathematics achievers of all races (2002). 
Others have agreed with these 
assessments, finding somewhat 
positive results of state-level ac- 
countability programs and little 
evidence of negative effects 
(Hanushek & Raymond, 2005; 

Jacob, 2005). These findings do 
not necessarily mean that NCLB 
will have positive effects on 
student achievement through its 
school accountability programs. 

Rather, the evidence shows that 
similar state-level accountability 
programs have had positive ef- 
fects, leveled inequalities in student opportunity to learn, 
and may be a step toward closing the achievement gap. 

NCLB school accountability is based on levels 
of student achievement — specifically, the proportion of 
students labeled “proficient.” But with the requirement of 
yearly testing in grades 3-8, many states have attempted to 
estimate school value-added to student achievement. This 
measure of change is fundamentally different from the 
measure of level currently used to determine AYP. Each 
indicator has its strengths and weaknesses. Proponents of 
level argue that equity is only served when all students 
reach high levels of achievement; gains in achievement are 
not acceptable if levels of achievement remain low. Propo- 
nents of value-added argue that accountability for levels 
of achievement is unfair to schools with large numbers 
of low-achieving students, as it holds them accountable 
for all the learning and inequity that occurred before they 
entered school. 

Recently, the Department of Education has allowed 
states to experiment with using value-added models of 
achievement. The potential problems described above are 
avoided by requiring that even value-added models must 
reach the 100% proficient goal by 2014. Abetter solution 
may be to use the two standards in concert, considering 
both level and value-added together. Schools doing well or 



poorly on both measures are easy to interpret; schools do- 
ing well on just one standard would have to be considered 
more carefully. This approach might make NCLB stronger 
by giving schools realistic targets to achieve. 

NCLB’s school accountability provisions have had 
some perhaps unanticipated effects on measures of student 
achievement. The act requires timely parental notification 
of school AYP status so school choice options can be ex- 
ercised. This requires rapid turn-around of achievement 
results following testing which in turn has pushed toward 
streamlined, multiple choice tests, even among states with 
historically strong commitments to performance assess- 
ment, such as Kentucky. At the same time, to calculate full 

value-added, spring testing dates 
have been pushed to as late in the 
year as possible. This conflict 
may be difficult to resolve. 

In 2006, the National 
Longitudinal Study and the 
Study of State Implementation 
of No Child Left Behind wrapped 
up the first phase of work (Le 
Lloch et al., 2006). The study’s 
primary goals were to examine 
the implementation of NCLB by 
states, districts, and schools, us- 
ing interviews of state officials in 
all 50 states and nationally representative samples of prin- 
cipals and teachers. The results lead to several important 
conclusions about the school accountability provisions of 
NCLB. In the 2003-2004 school year, 75% of schools and 
7 1 % of districts met AYP, with more elementary schools 
meeting AYP than middle or high schools. The larger size 
of middle and high schools, and the resulting requirement 
of disaggregated data, may account for this discrepancy. Of 
the schools that failed AYP, just under one-quarter, 23%, 
failed due to the performance of one group of students. This 
shows that most schools failing AYP are falling short for 
multiple groups of students. Diversity and size of schools 
also played a factor in AYP status — just 61% of schools 
held accountable for the progress of six or more groups 
of students met AYP, as opposed to 90% of schools with 
only one group. Perhaps surprisingly, 22% of principals 
in schools that failed AYP were not aware their school had 
failed AYP — evidence of a loosely coupled system. These 
findings point to several additional challenges of NCLB 
accountability implementation. Large schools with diverse 
student bodies seem especially likely to fall short of AYP. 
In the conclusion, we will discuss how AYP targets could 
be revised to address these and other concerns. 



...there is no evidence that 
schools can actually reach 100% 
proficient, and some scholars have 
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Student Accountability 

While student accountability is not directly cov- 
ered under NCLB, many states have some form of student 
accountability in their standards-based reform package, 
and more appear to be moving in that direction. Student 
accountability provisions might be considered in reautho- 
rization. 

Exit exams are a particularly popular type of student 
accountability, having been implemented in nineteen states 
and planned in seven more. This move toward exit exams 
comes despite NCLB’s measuring of graduation rate as 
one of its “other academic indicators.” While some are 
concerned about potential negative effects of exit exams 
on high school completion, the 
literature shows mixed but no 
large negative effects. In a study 
using a sample from the U.S. 

Census, Dee and Jacob (2007) 
found there was no relationship 
between dropout likelihood and 
introduction of an exit exam. Dee 
and Jacob also analyzed Minne- 
sota data and found that positive 
effects on dropout were greatest 
for low poverty districts and nega- 
tive effects were largest for high 

poverty districts and districts with high concentrations of 
black students. Alarmingly, the exit exam seems to have 
exacerbated the achievement gap. 

Highly Qualified Teachers 

Researchers investigating the distribution of 
qualified teachers since the implementation of NCLB 
have found that teacher quality is still more concentrated 
in low-poverty schools than in high-poverty schools (with 
“low-poverty schools” defined as schools with 10% or 
fewer students receiving free or reduced price lunch). 
Beyond these between-school differences, there are large 
within-school differences in teacher quality. An advan- 
taged student in a high-poverty school is still more likely 
to have a high-quality teacher than is a disadvantaged 
student in a low-poverty school (Desimone, Smith, & 
Frisvold, 2007). Thus far there have been only small and 
inconsistent changes in teacher quality in the first few years 
of NCLB. 

In the AIR/RAND study of implementation of 
NCLB, twenty percent of teachers did not know whether 
they were labeled highly qualified by the state. Among 
highly qualified high school teachers, 46% of English 
teachers and 59% of mathematics teachers lacked a content 
area degree (Le Floch et al., 2006). Yet the percentage of 
highly qualified teachers in most states is high — 97.2% 



in Missouri and Georgia, for instance. This suggests that 
the criteria for establishing highly qualified status should 
be defined more rigorously by states. At the same time, 
there are supply concerns to consider. Teacher quality is 
unequally distributed both within and between schools. 
How should teachers best be distributed to address the 
achievement gap? Taking high quality teachers from low- 
poverty schools and putting them in high-poverty schools 
is an idea that is unlikely to gain political traction. Setting 
a higher bar for teacher quality may shrink the absolute 
number of highly qualified teachers and concentrate them 
even more in low-poverty schools. Setting a low bar for 
teacher quality appears to accomplish nothing. The solu- 
tion to this problem is unclear, 
but solving it could be a key step 
toward realizing the goals of 
NCLB. 



Thus far there have been 
only small and inconsistent 
changes in teacher quality in the 
first few years of NCLB. 



Supplemental Services 

NCLB specifies that stu- 
dents attending schools failing 
AYP targets for more than one 
year in a row have certain rights. 
For starters, school districts must 
allow students to transfer from the 
failing school to another public 
school in the same district that has not been classified as 
“in need of improvement”. Also, if a school has fallen 
short of AYP for three years in a row, the school district 
must provide supplemental education services to students 
from low-income families. Both of these provisions have 
proven controversial. 

As for the provision of school choice, one thing is 
clear: very little choice is actually being exercised. This 
is the case not only because in many districts there are no 
schools to which students might transfer, but also because 
parents have often been informed of eligibility too late to 
make a reasonable transfer decision. The AIR/RAND study 
of NCLB found that just 15 states reported AYP status by 
the start of the school year and that 39% of districts re- 
quired to offer choice did not do so. As a result, of the 6.3 
million students eligible for choice, less than 1 % exercised 
this option (Le Floch et al., 2006). Even if choice were 
exercised by more families, the evidence is mixed as to 
the benefits for student achievement (Hill, 2007). Neither 
is there strong evidence, as some fear, that NCLB choice 
provisions would result in segregation. Regardless, with 
so few students currently exercising choice options the 
effects are certain to be small. 

Supplemental services generally take the form 
of small-group reading instruction delivered after school 
several days a week. Providers can be both certified teach- 





ers working through the school district and private not- 
for-profits. About a third as many students are eligible for 
this provision as for the choice provision. Among these 
approximately 2 million students, attendance rates are low 
and variable, with an average attendance of about 60 hours 
per year. Approximately 10 to 15% of eligible students 
participate (Farkas & Durham, 2007). Aside from concerns 
about the attendance rates (which will hopefully improve 
as the option becomes more stable), some are concerned 
that this format may not help alleviate the gap as it is in- 
tended. Perhaps encouraging competition among providers 
and limiting instruction to one-on-one instruction would be 
more likely to help close the poverty achievement gap. 

Suggestions for 
Reauthorization 

The evidence presented 
here is part of a growing body 
of research on the effects of 
NCLB and other accountability 
policies. From this analysis, 
several conclusions emerge. 

First, state implementations of 
NCLB are constantly changing 
and will probably continue to 
do so as the act matures. Since 2001, states have been 
forced to enact sweeping changes in a very short period of 
time. Some of these changes have not gone as planned, but 
as states become more accustomed to the accountability 
provisions of NCLB, it is likely that implementation will 
improve. This is already taking place in the more than 
twenty-live states that have revised their content stan- 
dards. 

Second, the evidence on the act’s effectiveness 
is limited but mostly neutral to positive. With regard to 
teacher turnover, while Clotfelter et al. (2004) found that 
North Carolina’s NCLB-like state accountability policies 
increased teacher turnover, Desimone et al. (2007) found 
no such evidence using a larger state database. Further- 
more, Desimone et al. found modest gains in teacher 
quality as a result of such policies, though the gains are 
not large enough to close the teacher quality gap. As for 
student retention, the act appears to have had no impact 
so far (Hauser, Frederick, & Andrew, 2007), though the 
evidence described above does suggest that students at the 
low end of the achievement distribution may not benefit 
from NCLB accountability. In all, there is little convinc- 
ing evidence that the potential negative effects that some 
feared have materialized. And while some are concerned 
about the loss of teacher autonomy, it is not clear that such 
a trend would necessarily be bad for student achievement 



(Porter, Floden, Freeman, Schmidt, & Schwille, 1988). 

Third, reauthorization may be difficult. The pub- 
lic is almost evenly split in terms of their opinion of the 
act and its effects on local schools. In a September, 2006 
poll conducted by Gallup and Phi Delta Kappa, 32% of 
respondents had a very or somewhat favorable view of the 
act, compared with 3 1 % who had a very or somewhat un- 
favorable view. Just over a quarter of respondents said the 
act was helping local schools, while 21% said it was hurt- 
ing (Rose & Gallup, 2006). However, most agree that the 
idea of accountability for education’s outputs as a means 
to close the achievement gaps is good. Reauthorization 
may depend on a tenuous political coalition in Congress 

(Loveless, 2007). 

If lawmakers are 
to reauthorize NCLB, a good 
template for that reauthoriza- 
tion is a theory created by Por- 
ter, Chester, and Schlesinger 
(2004) for how to do high- 
stakes testing right. The theory 
suggests three important steps: 
a) set a good target, b) make 
accountability symmetric, and 
c) make accountability fair. 
NCLB already stacks up fairly 
well when measured against these criteria, but there is 
room for improvement. 

Setting a Good Target 

Alignment of tests with ambitious content stan- 
dards in core academic subjects is one of the primary 
requirements of NCLB. The basis for this policy is the 
assumption that teachers will teach what is tested. What 
teachers teach is an important determinant of student 
achievement gains (Gamoran, Porter, Smithson, & White, 
1997). Specifically, NCLB requires content standards 
and testing in reading and math at all grades from third 
through eighth, with science required beginning in 2008. 
The act also requires that 95% of students be tested and 
that data be disaggregated by subgroups. These provisions 
show that the act sets a good target in terms of requiring 
accountability in several content areas at multiple grade 
levels and attempting to close the achievement gaps. 

At the same time, the variation in highly qualified 
teacher definitions, content standards, and tests is trou- 
bling, expensive, difficult to manage, and masking of real 
differences in achievement across states. Why should stu- 
dents be labeled differently in terms of their performance 
depending on the state in which they live? Why should 
students in some states have less qualified teachers than 
students in other states? Other than the federalist argument 



The time may have come to have 
an open debate about the possibility 
of national content and performance 
standards, national definitions of 
highly qualified teacher, and national 
achievement tests. 
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Commentary on Porter & Polikoff 

Robert Schwartz & Kathleen McCartney 
Harvard Graduate School of Education 

The Porter and Polikoff policy brief provides a balanced assessment of the strengths and limitations of 
the No Child Left Behind Act (NCLB) as well as a sensible set of recommendations for reauthorization. Porter 
and Polikoff remind us that NCLB was only enacted in 2002, and that some of its most important provisions 
are just now beginning to be implemented. Consequently, despite the fierce rhetoric on both sides, it is far too 
soon to come to a reasoned judgment about the success or failure of the law. Porter and Polikoff argue as much 
when they write that the evidence to date suggests that neither the worst fears of the critics nor the best hopes of 
NCLB’s advocates have yet been realized, a conclusion with which we concur. It might be more accurate to say 
that the data simply do not exist as yet to draw a causal conclusion about the effectiveness of NCLB. 

A second useful reminder concerns variation among states, especially with respect to the quality and 
rigor of standards, tests, and proficiency targets. Further, it is important to note that many states were already 
well-launched on a comprehensive standards-based reform agenda before the passage of NCLB, and that, in 
these states at least, it is virtually impossible to disentangle the effects of the federal law from the effects of state 
policies. 

Here we offer some additional policy observations based largely on our perspective from Massachusetts, 
a national leader in standards-based reform. Massachusetts passed a comprehensive reform program in 1993, 
provided substantial new funding targeted to high-need districts, and has managed to maintain a steady policy 
course for nearly 15 years despite the periodic efforts of the teacher organizations and others to weaken the law. 
The data are encouraging. Student performance across grades has risen over the last several years, an impres- 
sive feat, given that the Massachusetts state test is generally regarded as one of the most rigorous in the nation. 
Student gains on the Massachusetts state assessments have been accompanied by similar gains on the National 
Assessment of Education Progress (NAEP), where Massachusetts now leads the nation. This suggests that gains 
in state tests are not simply a consequence of increased familiarity with the tests or more attention to test prepa- 
ration. Despite this progress, there remains a deeply troubling racial achievement gap in Massachusetts that is 
confounded with SES; for example, 35% more white students score in the in the proficient or advanced cat- 
egories, compared with black students, and 38% more white students scores in these categories compared with 
Hispanic students. 

We argue that accountability must be part of a larger school reform effort. While accountability may 
lead to an overall increase in test results over time, it is unlikely to close achievement gaps with respect to race 
and SES for two reasons. First, children of color and children from families with fewer economic resources 
enter kindergarten already behind their peers. Second, school resources are confounded with race and SES. To 
address this challenge, Massachusetts is focusing on two interrelated strategies: early childhood education for 
all, and extended learning time, especially for schools serving high-need children. These are important initia- 
tives, likely to yield important gains in achievement, because they each focus on additional instructional time to 
promote learning. Education research has documented the success of these strategies, along with others re- 
viewed by Porter and Polikoff, such as smaller class sizes and teacher quality efforts. States will need additional 
resources to support a comprehensive school reform effort to support the learning of all children, ensuring none 
is left behind. 
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Commentary on Porter and Polikoff 



David N. Figlio 

University of Florida and National Bureau of Economic Research 

Porter and Polikoff present a fair-minded discussion of many of the issues on the table and thought- 
ful suggestions for NCLB’s reauthorization. The authors are careful to not place much emphasis on the early 
evidence regarding NCLB’s effectiveness, and even more caution is warranted. Because of the lack of a counter- 
factual, it is impossible to draw conclusions about the success of NCLB based on temporal changes in aggregate 
student performance. Therefore, the best evidence on the potential effects of school-based accountability come 
from analysis of state-level accountability policies. Here, a general consensus is emerging that schools facing 
accountability pressure have raised test scores in the tested subjects, at least in math. 

However, numerous authors have found that schools facing accountability pressure tend to improve test 
scores on higher-stakes exams but not necessarily on lower-stakes exams, leading some to complain that ac- 
countability systems encourage “teaching to the test.” Others, however, argue that teaching to the test is a desir- 
able outcome of accountability - provided that the test is broad-based, rigorous and closely aligned to objective 
and appropriate standards. But other potential incentive effects are harder to justify on educational grounds: 
Researchers have discovered that schools often respond to accountability pressure by targeting their attention 
on select groups of students (Neal and Schanzenbach, 2007), excluding potentially low-performing students 
from the testing pool either through selective reclassification as disabled (Cullen and Reback, 2006) or through 
strategic deployment of discipline policy (Figlio, 2006), or even manipulating school meals to give students 
short-term brain boosts for tests (Figlio and Winicki, 2005). Since school accountability encourages schools to 
both genuinely improve and artificially inflate measured performance, it is difficult to know how to interpret the 
measured test performance that results. 

School accountability systems based on “value added” measures of assessment present schools with 
dramatically fewer incentives to “game the system,” and therefore have the potential to provide more accurate 
views of schools’ true contributions. This, combined with the authors’ points regarding incentives to concentrate 
on a broader group of students and inherent fairness issues, suggests that the U.S. Department of Education’s 
willingness to give states such as Arkansas, Delaware, Florida, North Carolina and Tennessee the authority to 
introduce value added elements into the basic approach for calculating NCLB progress is a very positive step 
that should be continued and expanded under a reauthorization of NCLB. 

But value added measurements are not a silver bullet. Kane and Staiger (2002) point out that test scores 
are inherently measured with error, and measurement problems are particularly extreme for smaller schools. 
Value added measures of student progress geometrically exacerbate the measurement error problem, reducing 
the reliability of the resulting measures. This is particularly true if one wants to focus on specific subgroups - a 
key component of NCLB. Value added measures may improve fairness and reduce incentives for deleterious 
behavior, but they do so at the cost of reducing the ability to monitor the progress of small groups. And since 
test score gains tend to be greater for more advantaged students (Ladd and Walsh, 2002), schools may have an 
incentive under a value added system to focus on the more advantaged students rather than students below profi- 
ciency levels. 

It is therefore not clear which approach best serves the needs of the students traditionally left behind. 
That said, an ideal system would likely be a hybrid of traditional status-based measures and value added mea- 
sures, coupled with increased flexibility to meet targets, extra attention paid to the learning gains made by 
historically low-performing students, and, not least, additional funding to help schools provide the necessary 
remediation to low-performing students in order to maximize their chances of achieving proficiency. 
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in favor of states’ rights, it is hard to see a strong argument 
in favor of widely different state standards. The time may 
have come to have an open de- 
bate about the possibility of na- 
tional content and performance 
standards, national definitions 
of highly qualified teacher, and 
national achievement tests. 

National standards and tests 
would simplify the provisions 
of curricular materials and pro- 
fessional development. They 
would also increase efficiency 
and reduce redundancy. At 
least three relevant bills have 
been introduced in Congress 
addressing these topics. A bill 
introduced by Senator Clinton 
would introduce voluntary 
national standards and effective teaching methods in 
math and science from grades K through 12 (“National 
Mathematics and Science Consistency Act,” 2007). An- 
other, introduced by Senator Dodd, would create rigorous 
voluntary math and science content standards in grades K 
through 12 and align the standards to tests of achievement 
(“SPEAK Act,” 2007). A third act calls for alignment 
studies comparing state standards to NAEP and grants 
to improve alignment (“SUCCESS Act,” 2007). Clearly, 
legislators are thinking about national content standards. 
Any debate on the subject will be fierce, but the time for 
that debate is here. 

Also, it is clear that NCLB targets for AYP must 
be revised. There is no proof that states can achieve the 



100% proficient target — and there are predictions to the 
contrary. While high standards for student achievement 

are admirable, standards must 
be reasonable to be relevant. 
Targets for AYP should also 
be revised to include a value- 
added component. This should 
ensure that schools focus on 
all students, not just those at 
the brink of proficiency. With 
these changes, NCLB will set 
better targets for high-stakes 
accountability. 

Symmetry 

Symmetry refers 
to holding all parts of the sys- 
tem — schools and students — 
accountable for student learn- 
ing. NCLB is not symmetric, as it does not require holding 
students accountable. This imbalance has important im- 
plications — education tends to work best when everyone 
works together. This may not be the case under NCLB, 
if the commonly cited reports of students not trying their 
best on NCLB tests are to be believed. Student account- 
ability has traditionally taken the form of grade-to-grade 
promotion or exams, but other, less severe measures may 
be equally effective. For example, students’ scores could be 
included on transcripts or count toward grades. It would be 
ideal if students were intrinsically motivated, but it is likely 
that some form of weall-thought out student accountability 
measures may provide needed external motivation. 



It would be ideal if students were 
intrinsically motivated, but it is likely 
that some form of well-thought out 
student accountability measures may 
provide needed external motivation. 
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Fairness 

There are three components to fairness. The first 
two refer to support; both schools and students must be 
provided the support they need. For students, this means 
adequate opportunity to learn, something which may be 
critical for closing the achievement gaps. For schools, 
this means providing needed resources. Money alone is 
not a solution, but NCLB provides supplemental services 
for students and state and district services for schools in 
need of improvement. The third component of fairness 
is the accuracy and validity of data. NCLB requires test 
alignment to standards, an important component of valid- 
ity. However, states are struggling to create high quality 
student assessments. 



...there are improvements that 
can and should be made to the act as 
reauthorization is considered. 



To improve the fairness of NCLB, changes in 
provisions such as supplemental services and highly quali- 
fied teachers may be necessary. Highly qualified teacher 
standards are more variable than they should be, and a dis- 
cussion of standardization of requirements may be needed. 
The strength of such a standard would be tricky — too high, 
and many teachers will fall short; too low, and the standard 
will be meaningless. Supplemental service requirements 
might be changed to require more effective one-on-one 
tutoring. 

NCLB is an important, timely topic for education 
researchers and policy-makers. It affects students, teachers, 
and schools in significant ways on a daily basis. It does not 
appear to be having the profound effects on achievement 
that were hoped, but there are some signs that gains are 
being made. Still, there are improvements that can and 
should be made to the act as reauthorization is considered. 
There is good and relevant education research that should 
inform the deliberations. 
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